381 research outputs found
Characterizations of Super-regularity and its Variants
Convergence of projection-based methods for nonconvex set feasibility
problems has been established for sets with ever weaker regularity assumptions.
What has not kept pace with these developments is analogous results for
convergence of optimization problems with correspondingly weak assumptions on
the value functions. Indeed, one of the earliest classes of nonconvex sets for
which convergence results were obtainable, the class of so-called super-regular
sets introduced by Lewis, Luke and Malick (2009), has no functional
counterpart. In this work, we amend this gap in the theory by establishing the
equivalence between a property slightly stronger than super-regularity, which
we call Clarke super-regularity, and subsmootheness of sets as introduced by
Aussel, Daniilidis and Thibault (2004). The bridge to functions shows that
approximately convex functions studied by Ngai, Luc and Th\'era (2000) are
those which have Clarke super-regular epigraphs. Further classes of regularity
of functions based on the corresponding regularity of their epigraph are also
discussed.Comment: 15 pages, 2 figure
Activity Identification and Local Linear Convergence of Douglas--Rachford/ADMM under Partial Smoothness
Convex optimization has become ubiquitous in most quantitative disciplines of
science, including variational image processing. Proximal splitting algorithms
are becoming popular to solve such structured convex optimization problems.
Within this class of algorithms, Douglas--Rachford (DR) and alternating
direction method of multipliers (ADMM) are designed to minimize the sum of two
proper lower semi-continuous convex functions whose proximity operators are
easy to compute. The goal of this work is to understand the local convergence
behaviour of DR (resp. ADMM) when the involved functions (resp. their
Legendre-Fenchel conjugates) are moreover partly smooth. More precisely, when
both of the two functions (resp. their conjugates) are partly smooth relative
to their respective manifolds, we show that DR (resp. ADMM) identifies these
manifolds in finite time. Moreover, when these manifolds are affine or linear,
we prove that DR/ADMM is locally linearly convergent. When and are
locally polyhedral, we show that the optimal convergence radius is given in
terms of the cosine of the Friedrichs angle between the tangent spaces of the
identified manifolds. This is illustrated by several concrete examples and
supported by numerical experiments.Comment: 17 pages, 1 figure, published in the proceedings of the Fifth
International Conference on Scale Space and Variational Methods in Computer
Visio
Linear Superiorization for Infeasible Linear Programming
Linear superiorization (abbreviated: LinSup) considers linear programming
(LP) problems wherein the constraints as well as the objective function are
linear. It allows to steer the iterates of a feasibility-seeking iterative
process toward feasible points that have lower (not necessarily minimal) values
of the objective function than points that would have been reached by the same
feasiblity-seeking iterative process without superiorization. Using a
feasibility-seeking iterative process that converges even if the linear
feasible set is empty, LinSup generates an iterative sequence that converges to
a point that minimizes a proximity function which measures the linear
constraints violation. In addition, due to LinSup's repeated objective function
reduction steps such a point will most probably have a reduced objective
function value. We present an exploratory experimental result that illustrates
the behavior of LinSup on an infeasible LP problem.Comment: arXiv admin note: substantial text overlap with arXiv:1612.0653
From error bounds to the complexity of first-order descent methods for convex functions
This paper shows that error bounds can be used as effective tools for
deriving complexity results for first-order descent methods in convex
minimization. In a first stage, this objective led us to revisit the interplay
between error bounds and the Kurdyka-\L ojasiewicz (KL) inequality. One can
show the equivalence between the two concepts for convex functions having a
moderately flat profile near the set of minimizers (as those of functions with
H\"olderian growth). A counterexample shows that the equivalence is no longer
true for extremely flat functions. This fact reveals the relevance of an
approach based on KL inequality. In a second stage, we show how KL inequalities
can in turn be employed to compute new complexity bounds for a wealth of
descent methods for convex problems. Our approach is completely original and
makes use of a one-dimensional worst-case proximal sequence in the spirit of
the famous majorant method of Kantorovich. Our result applies to a very simple
abstract scheme that covers a wide class of descent methods. As a byproduct of
our study, we also provide new results for the globalization of KL inequalities
in the convex framework.
Our main results inaugurate a simple methodology: derive an error bound,
compute the desingularizing function whenever possible, identify essential
constants in the descent method and finally compute the complexity using the
one-dimensional worst case proximal sequence. Our method is illustrated through
projection methods for feasibility problems, and through the famous iterative
shrinkage thresholding algorithm (ISTA), for which we show that the complexity
bound is of the form where the constituents of the bound only depend
on error bound constants obtained for an arbitrary least squares objective with
regularization
- …